31/10/24 - Demo and admin

Did demo.

Expenses for San Diego.

30/10/24 - Demo & Gatekeeper

Debugging helm installation of gatekeeper

Finishing demo.

Also demo prep.

Reviewing gatekeeper code/docs.

Fixing some bits of the DM API, implementing the actual staging logic.

Making the Gatekeeper helm chart work with JC.

Refactoring better for token timeouts (created AuthenticatedSession).

Made DM API more robust to retries & token timeouts

Explored what would need to be done for staging:

route checks storage_area_id is tier0 RSE
gets site of this RSE
checks prepareData service exists at same site
runs chained move/prepare data task -- prepare_data task should accept did & RSE to infer the rule_id, as well as info about the web service to prepare the data and the user's token (and stratgegy?) -- checks status of rule id before proceeding (must be OK) -- "unpacks" the contents of the container DID -> refer to move_data structure (.containers, .datasets, .files) (must conform?) -- runs web service for preparing (feed in USER token, data files with structure & job id) + polls -- need to add "mock" data preparation service to docker-compose for testing?

Need to write the permissions service plugin for prepareData requests.

Finessing mov data task, added skeleton for staging.

Moved movement of data to celery task so can chain move & prepare together for staging.

Verified works for move, need to add staging endpoints back in and think about the interface.

Backlog refinement meeting.

Ingestion meeting.

Looking at gatekeeper (refamiliarising myself with it)

ITSRC added, JPSRC checked for readiness.

Some feature level support (dashboards, ingestion dashboards, rse naming)

Travel expenses. ITSRC, JPSRC, CHSRC support.

Some feature level support (dashboards).